Papers with downstream classification tasks
Token Prepending: A Training-Free Approach for Eliciting Better Sentence Embeddings from LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on prompt engineering to extract sentence embeddings from large language models (LLMs) but these models are mostly decoder-only and the earlier tokens in the sentence cannot attend to the latter, resulting in biased encoding of sentence information and cascading effects on the final decoded token. |
| Approach: | They propose a plug-and-play and training-free technique that prepends each layer’s decoded sentence embedding to the beginning of the sentence in the next layer’ s input. |
| Outcome: | The proposed technique can significantly improve the performance of existing prompt-based sentence embedding methods across different LLMs while incurring negligible additional inference cost. |
Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on prompt engineering to encode the full semantics of a sentence into the embedding of the last token. |
| Approach: | They propose a technique that introduces an extra auxiliary prompt to elicit better sentence embedding . they propose to use the hidden state of the token as the sentence embedded in LLMs . |
| Outcome: | The proposed technique can improve performance of existing prompt-based methods on STS tasks and downstream classification tasks. |
Attentively Embracing Noise for Robust Latent Representation in BERT (2020.coling-main)
Copied to clipboard
| Challenge: | Modern digital personal assistants interact with users through voice . high error rates still prevail in the widespread adoption of speech technology . |
| Approach: | They propose to extract more robust latent representations for noisy ASR text classification using transformer tokens and attentive embracement layer and multi-head attention layer. |
| Outcome: | The proposed model significantly outperforms the baseline model on the Chatbot and Snips corpora for intent classification with ASR error. |
Efficient Sentence Embedding using Discrete Cosine Transform (D19-1)
Copied to clipboard
| Challenge: | Modern NLP systems rely on word embeddings as input units to encode statistical semantic and syntactic properties of words. |
| Approach: | They propose to use discrete cosine transform to compress word sequences in order-preserving manner. |
| Outcome: | The proposed model preserves syntactic information in semantic probing tasks . it is comparable to vector averaging but mediocre in performance. |
On The Performance of Time-Pooling Strategies for End-to-End Spoken Language Identification (2020.lrec-1)
Copied to clipboard
| Challenge: | Language identification (LID) from speech is commonly tackled using similar approaches to those employed for speaker verification/recognition. |
| Approach: | They propose to combine local descriptors and global descriptores into a single global description that can be used for downstream classification tasks. |
| Outcome: | The proposed methods outperform well-known benchmark systems and previously results based on attention only. |
SHONGLAP: A Large Bengali Open-Domain Dialogue Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing open-domain dialogue systems suffer from data scarcity due to unavailability of high-quality datasets for low-resource languages like Bengali. |
| Approach: | They propose to prepare large-scale open-domain dialogue datasets from podcasts and talk-shows and label them based on weak-supervision techniques. |
| Outcome: | The proposed corpus improves performance of large language models in case of downstream classification tasks during fine-tuning. |
Encouraging Paragraph Embeddings to Remember Sentence Identity Improves Classification (P19-1)
Copied to clipboard
| Challenge: | Existing paragraph embedding methods do not capture basic linguistic properties, but their performance is limited. |
| Approach: | They propose a paragraph embedding method that can't tell whether a sentence occurs in a given paragraph. |
| Outcome: | The proposed method outperforms reconstruction-based methods on a semi-supervised dataset and improves on benchmark datasets. |
DiSec: Mitigating Backdoors in Pre-trained Language Models via Disentanglement of Adversarial Weights for Secure Fine-Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing defenses rely on privileged assumptions, limiting their applicability in realistic settings. |
| Approach: | They propose a task-agnostic backdoor attack that contaminates pre-trained language models . authors propose auxiliary text purification framework that uses only clean auxiliary data . |
| Outcome: | The proposed framework suppresses attack success while preserving clean-task utility. |